Papers with intent detection
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. |
| Approach: | They propose a framework that combines the scalability of LLM-generated labels with the precision of human annotations to achieve higher speed and accuracy comparable to larger models. |
| Outcome: | The proposed framework significantly improves accuracy across utterance-level dialogue tasks, including sentiment detection (over 2%), dialogue act classification (over 1.5%), etc. |
Copied to clipboard
| Challenge: | a problem faced by conversational agents working with large documents is the frequent presence of information that is irrelevant to the agent. |
| Approach: | They propose a neural model for scoping relevant information from a large document . they show that the model performs better with emails than existing baselines . |
| Outcome: | The proposed model improves intent detection and entity extraction tasks without drop in recall. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) produce ambiguous examples with regard to untargeted classes. |
| Approach: | They propose to use a sentence transformer to detect ambiguous augmented examples generated by Large Language Models for intent recognition. |
| Outcome: | The proposed method improves the quality of augmented data generated by large language models in low-resource scenarios. |
Copied to clipboard
| Challenge: | Current intent detection work experiments with minor intent categories. |
| Approach: | They propose a retrieval-augmented generation framework that uses query-to-query and query- to-metadata approaches to retrieve intents from metadata. |
| Outcome: | The proposed framework improves on query-to-query (Q2Q) and query- to-metadata (Q 2M) approaches. |
Copied to clipboard
| Challenge: | Instruction tuning is emerging in NLP, but has not been explored for dialogue-related tasks. |
| Approach: | They propose an instruction tuning framework for dialogue that leverages natural language instructions with language models to induce zero-shot generalization on unseen tasks. |
| Outcome: | The proposed framework enables good zero-shot performance on unseen datasets and tasks such as dialogue evaluation and intent detection. |
Copied to clipboard
| Challenge: | Existing methods to detect toxic behavior in online gaming environments are limited by utterance-level annotation. |
| Approach: | They propose to annotate game chat utterances for toxicity detection through intent classification and slot filling. |
| Outcome: | The proposed model improves the detection of toxic speech in online gaming environments and reveals limitations of current models. |
Copied to clipboard
| Challenge: | Intent detection and slot filling are two main tasks for building a spoken language understanding system. |
| Approach: | They propose to use a sequence to sequence model to generate both intent and slot filling tasks together to perform the two tasks jointly. |
| Outcome: | The proposed model achieves 0.5% intent accuracy improvement and 0.9 % slot filling improvement on the ATIS benchmark data. |
Copied to clipboard
| Challenge: | Existing studies have focused on disfluency detection and removal, with limited studies into its impact on downstream tasks. |
| Approach: | They propose to incorporate disfluency in summarization models to reduce the impact of replacement disfluencies on natural language processing tasks. |
| Outcome: | The proposed model improves on both public and real-life datasets and shows that it can handle disfluent data with up to 6.99-point degradation in Rouge-L score and replacement disfluencies have the highest negative impact. |
Copied to clipboard
| Challenge: | Neural network models have gained traction for sentence-level intent classification and token-based slot-label identification. |
| Approach: | They propose a neural network model that performs multi-label classification for identifying multiple intents and produces token-based slot-l labels at the token-level. |
| Outcome: | The proposed model provides a small but statistically significant improvement on the ATIS dataset and 55% accuracy improvement on an internal multi-intent dataset. |
Copied to clipboard
| Challenge: | Recent research shows promising results by jointly learning of slot filling and intent detection tasks. |
| Approach: | They propose a way to combine slot filling and slot filler learning to achieve state-of-the-art results. |
| Outcome: | The proposed model outperforms existing methods on benchmark datasets and ATIS datasets. |
Copied to clipboard
| Challenge: | RiSAWOZ contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues spanning over 12 domains . despite of substantial progress made, there are challenges in creating challenging datasets in terms of size, multiple domains, semantic annotations and complexity. |
| Approach: | They propose a large-scale multi-domain Chinese Wizard-of-Oz dataset with rich semantic annotations that captures discourse phenomena for task-oriented dialogue modeling. |
| Outcome: | The proposed dataset contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues with more than 150K utterances spanning over 12 domains. |
Copied to clipboard
| Challenge: | Pretrained language models perform structural understanding tasks that focus on understanding one aspect of the text. |
| Approach: | They propose a method for improving the structural understanding abilities of language models by pretraining them to generate structures from the text on task-agnostic corpora. |
| Outcome: | The proposed model performs state-of-the-art on 21 of 28 datasets. |
Copied to clipboard
| Challenge: | Existing methods to identify all possible user intents at design time are expensive and require storage of past data. |
| Approach: | They propose to continually train an intent detector on new intents while maintaining performance on prior intents. |
| Outcome: | The proposed method outperforms exemplar replay-based approaches on lifelong intent detection tasks and achieves state-of-the-art on four public datasets. |
Copied to clipboard
| Challenge: | Existing Transformer-based language models (LMs) are not effective as sentence encoders when used off-the-shelf. |
| Approach: | They propose a method which turns a pretrained LM into a universal conversational encoder and task-specialised sentence encoder. |
| Outcome: | The proposed framework achieves state-of-the-art ID performance across the board with particular gains in the most challenging, few-shot setups. |
Copied to clipboard
| Challenge: | Existing models for slot filling and intent detection fail to fully utilize cooccurrence relations between slots and intents, which restricts their potential performance. |
| Approach: | They propose a novel Collaborative Memory Network (CM-Net) that captures slot-specific and intent-specific features in a collaborative manner. |
| Outcome: | The proposed network outperforms existing models on two benchmarks and a self-collected corpus. |
Copied to clipboard
| Challenge: | Intent detection is a fundamental element in task-oriented dialogue systems, usually occurring within the Natural Language Understanding component. |
| Approach: | They propose an in-context data augmentation approach that fine-tunes a pre-trained language model and synthesizes new datapoints that correspond to given intents. |
| Outcome: | The proposed method produces training data that achieves state-of-the-art on three challenging intent detection datasets and performs on par with the state- of-the art in full-shot settings. |
Copied to clipboard
| Challenge: | Prior work has shown that intent detection enhances LLMs’ moderation guardrails, but the robustness of these guardrail mechanisms under malicious manipulations remains under-explored. |
| Approach: | They propose a two-stage intent-based prompt-refinement framework that first transforms harmful inquiries into structured outlines and further reframes them into declarative-style narratives. |
| Outcome: | The proposed framework outperforms several cutting-edge jailbreak methods and evades even advanced Intent Analysis (IA) and Chain-of-Thought (CoT)-based defenses. |
Copied to clipboard
| Challenge: | Traditional approaches to intent detection struggle with out-of-scope (OOS) detection. |
| Approach: | They propose to use adaptive in-context learning and chain-of-thought prompting to detect intent in SOTA LLMs. |
| Outcome: | The proposed system achieves 2% of native accuracy with 50% less latency. |
Copied to clipboard
| Challenge: | In recent years, the representation of words as vectors in a vector space has gained a high degree of attention in the research community. |
| Approach: | They introduce a new language resource that represents dialogue utterances in vector space and captures the semantic meaning of those utterrances in the dialogue context. |
| Outcome: | The proposed model captures relevant semantic information by comparing them to manually annotated dialogue acts. |
Copied to clipboard
| Challenge: | a lack of research on multilingual or cross-lingual task-oriented dialog systems has limited results . we propose a zero-shot adaptation of task-orientated dialog systems to low-resource languages . task-focused systems are often trained with monolingual datasets that are expensive to build or acquire . |
| Approach: | They propose a zero-shot adaptation of multilingual task-oriented dialog systems to low-resource languages using latent variables and a set of very few parallel word pairs. |
| Outcome: | The proposed model performs better in natural language understanding task compared to state-of-the-art model . the proposed model uses very few parallel word pairs to refine cross-lingual representations . |
Copied to clipboard
| Challenge: | Out-of-Domain (OOD) Intent Classification and New Intent Discovering are two tasks in the Task-Oriented Dialogue System. |
| Approach: | They propose a task paradigm to extend Out-of-Domain (OOD) Intent Classification and New Intent Discovering tasks in the Task-Oriented Dialogue System. |
| Outcome: | The proposed scheme improves on existing OOD intent classification and discovery datasets. |
Copied to clipboard
| Challenge: | Text embeddings are an essential building component of several NLP tasks. |
| Approach: | They propose a regional expansion of MTEB covering 59 languages, 14 tasks, and 38 datasets, including six newly added datasets. |
| Outcome: | The proposed model outperforms baselines and mE5 in hate speech detection, intent detection, and emotion classification tasks. |
Copied to clipboard
| Challenge: | Experimental results show that the combination of regular expressions and NNs improves learning effectiveness when a small number of training examples are available. |
| Approach: | They propose to combine a neural network (NN) with regular expressions (RE) to improve supervised learning for NLP by exploiting the rich expressiveness of REs at different levels within a NN. |
| Outcome: | The proposed approach significantly improves learning effectiveness when a small number of training examples are available. |
Copied to clipboard
| Challenge: | Task-oriented dialogue systems are typically constructed for a single domain or language and do not generalise well beyond this. |
| Approach: | They constructed a multilingual, multi-intent, multi domain dataset to support work on Natural Language Understanding (NLU) in ToD across multiple languages and domains simultaneously. |
| Outcome: | The proposed dataset extends the English-only dataset to include manual translations into a range of high, medium, and low resource languages in two domains (banking and hotels). |
Copied to clipboard
| Challenge: | Existing approaches to filling slots that take on values from a virtually unlimited set have been lacking in the natural language area. |
| Approach: | They propose a new attention-based recurrent neural network (RNN) model that captures the concept: Understanding the role of a word may vary according to how long a reader focuses on a particular part of . sentence. |
| Outcome: | The proposed model outperforms existing models with respect to discovering ‘open-vocabulary’ slots without any external information, such as a named entity database or knowledge base. |
Copied to clipboard
| Challenge: | Existing few-shot learning methods learn a single task each time with a few examples . but, real-world applications often contain multiple closely related tasks . |
| Approach: | They propose a few-shot joint learning scheme that captures intent and slot relationships from only a handful of examples and adapts the bridged metric space to specific few- shot domain. |
| Outcome: | The proposed model outperforms baseline models on two public datasets on intent and slot . the proposed model significantly outperformed baseline models in one and five shots settings. |
Copied to clipboard
| Challenge: | Existing models for natural language understanding are based on a well-defined intent 1 ontology. |
| Approach: | They propose to retrain the natural language understanding model as new data from real users are merged into existing data. |
| Outcome: | The proposed model shows that the semantically entangled intents can be recognized with an automatic workflow. |
Copied to clipboard
| Challenge: | Existing methods for pretraining language models do not consider spoken language properties. |
| Approach: | They propose a framework that trains neural lattice language models to provide contextualized representations for spoken language understanding tasks. |
| Outcome: | The proposed framework outperforms baselines on spoken inputs on intent detection and dialogue act recognition datasets. |
Copied to clipboard
| Challenge: | Existing methods to classify intents are labor-intensive and time-consuming as intents will be diverse and new intents may be involved. |
| Approach: | They propose a zero-shot intent detection problem which aims to detect emerging user intents where no labeled utterances are currently available. |
| Outcome: | The proposed model can discriminate emerging intents when no labeled utterances are available in training data. |
Copied to clipboard
| Challenge: | Low-resource languages and computational expenses pose significant challenges in the domain of large language models. |
| Approach: | They propose a novel approach that uses adversarial techniques to mitigate the impact of language-specific information in contextual embeddings generated by large multilingual language models. |
| Outcome: | The proposed approach excels in zero-shot scenarios for Latin languages like Spanish, but fails to perform for languages distant from English, such as Thai and Persian. |
Copied to clipboard
| Challenge: | Existing methods to train task-oriented dialogue systems in monolingual datasets are expensive to build. |
| Approach: | They propose a hierarchical framework to classify intents in high-level and slot filling in low-level . they incorporate sentence-level alignment among different languages to enhance intent detection . |
| Outcome: | The proposed framework achieves the performance on a public task-oriented dialog dataset. |
Copied to clipboard
| Challenge: | pixel-based models can be used to transfer learning from standard languages to dialects . pretrained language models achieve strong results for languages seen during training, but their performance declines with out-of-domain dialects. |
| Approach: | They compare pixel-based models to token-based ones to evaluate dialects . standard german is tokenized in a more meaningful way, whereas the Bern dialect is tokenize in pixel form . |
| Outcome: | The proposed models outperform token-based models in part-of-speech tagging, dependency parsing and intent detection for zero-shot dialect evaluation by up to 26 percentage points in some scenarios, though not in Standard German. |
Copied to clipboard
| Challenge: | Existing joint models for intent detection and slot filling show insufficient robustness . however, some small changes of inputs can fool the models to produce wrong predictions . |
| Approach: | They propose a joint adversarial training model that generates adversarials to attack the joint model and trains the model to defend against the adversarial examples. |
| Outcome: | The proposed model achieves significantly higher scores and improves robustness on two datasets. |
Copied to clipboard
| Challenge: | Spatial domain queries have unique properties making them more challenging for language understanding than common conversational queries. |
| Approach: | They propose a language understanding framework for spatial domain queries that jointly learns the intent detection and entity linking tasks on a voice assistant service. |
| Outcome: | The proposed framework outperforms baseline methods with a significant margin. |
Copied to clipboard
| Challenge: | Slot-filling and intent detection tasks are well-established tasks in Conversational AI, but current benchmarks for these tasks rely on evaluations of low-resource languages and translations from English benchmarks. |
| Approach: | They propose to use a multilingual, open-source benchmark dataset for 16 African languages with utterances generated by native speakers across diverse domains. |
| Outcome: | The proposed dataset compares multilingual transformer models and prompting large language models (LLMs) with the English language. |
Copied to clipboard
| Challenge: | Existing studies on text-image content have focused on image as primary content, and text as secondary content. |
| Approach: | They propose a multimodal dataset of 1299 Instagram posts labeled for three orthogonal taxonomies . they show that employing both text and image improves intent detection by 9.6 . |
| Outcome: | The proposed model shows that using both text and image improves intent detection by 9.6 compared to using only the image modality. |
Copied to clipboard
| Challenge: | Existing models for slot filling and intent detection lack bi-directional interrelated connections between the intent and slots. |
| Approach: | They propose a bi-directional interrelated model for slot filling and intent detection that uses an SF-ID network to establish direct connections between the two tasks to promote each other mutually. |
| Outcome: | The proposed model improves on ATIS and Snips datasets in sentence-level semantic frame accuracy and improves performance on the two tasks. |
Copied to clipboard
| Challenge: | Existing models with quadratic time and memory complexity restrict input length . however, analyzing extensive sequential contexts is challenging . |
| Approach: | They propose a neural network architecture that captures contextual dependencies in linear time and a nonlinear readout to model short-term dependencies within sentences. |
| Outcome: | The proposed model outperforms baseline models on EmoryNLP datasets and on IEMOCAP and MultiWOZ datasets. |
Copied to clipboard
| Challenge: | Existing approaches to intent detection assume that each utterance represents only a single intent. |
| Approach: | They propose a framework for intent detection that can learn multiple representations of a given user utterance under the context of different intent labels in an optimized semantic space. |
| Outcome: | The proposed framework achieves state-of-the-art on multiple public benchmark datasets and a private real-world dataset for the multi-intent detection task. |
Copied to clipboard
| Challenge: | Current efforts to bridge the two modes of interaction are reactive, focusing on responding to user inputs rather than coordinating dialogue flows. |
| Approach: | They propose a dataset designed for transition-aware dialogue modeling that incorporates structurally diverse and integrated mode flows. |
| Outcome: | The proposed dataset outperforms baseline models in intent detection and mode transition handling. |
Copied to clipboard
| Challenge: | Existing data augmentation methods rely on few labelled examples for each intent category, which can be expensive in settings with many possible intents. |
| Approach: | They propose a data augmentation method for intent detection in zero-resource domains by using an open-source large language model and a smaller sequence-to-sequence model. |
| Outcome: | The proposed method significantly improves the data utility and diversity over the zero-shot LLM baseline for unseen domains and over common baseline approaches. |
Copied to clipboard
| Challenge: | Existing research focuses on simple queries with a single intent, lacking effective systems for handling complex queries with multiple intents. |
| Approach: | They propose a multi-label multi-class intent detection dataset curated from existing benchmarks and a pointer network-based architecture to extract intent spans and detect multiple intents with coarse and fine-grained labels in the form of sextuplets. |
| Outcome: | The proposed system outperforms baseline approaches in terms of accuracy and F1-score. |
Copied to clipboard
| Challenge: | Existing methods for OOD intent detection are limited to single dialogue turns. |
| Approach: | They propose a context-aware OOD intent detection framework to model multi-turn contexts in OOD context detection tasks using unlabeled data. |
| Outcome: | The proposed framework improves the F1-OOD score by 29% on multi-turn OOD detection tasks compared to the previous best method. |
Copied to clipboard
| Challenge: | generative AI is expanding in education, yet empirical analyses of large-scale and real-world interactions between students and AI systems remain limited. |
| Approach: | They present a dataset based on a semester-long experiment with 212 college students in English as Foreign Language (EFL) writing courses. |
| Outcome: | The proposed dataset includes conversation logs, students’ intent, students' self-rated satisfaction, and students’ essay edit histories. |
Copied to clipboard
| Challenge: | a new study examines the effectiveness of large language models and non-LLMs in multimodal intent detection . large-scale multimodal data integrations include text, audio, and visual inputs . |
| Approach: | They propose a framework to debias multimodal intent detection datasets by using human evaluation. |
| Outcome: | The proposed framework debiases the datasets and shows that mistral-7B outperforms most competitive models by approximately 9% on MIntRec-1 and 4% on MIndRec2.0. |